Fostering methodological reasoning in pre-service mathematics teachers: A hypothetical learning trajectory for quasi-experimental design
##plugins.themes.bootstrap3.article.sidebar##
##plugins.themes.bootstrap3.article.main##
Abstract
Quasi-experimental designs (QEDs) are widely used in educational research, however pre-service mathematics teachers often struggle to justify their methodological choices when ideal experimental conditions cannot be met. This study designed and refined a pedagogical approach to support mathematics education students’ conceptual understanding of QEDs. Employing educational design research, the study involved 41 third-year undergraduates enrolled in an experimental design in education course. The intervention was conducted across two iterative cycles: a teaching experiment (n = 9) and a classroom experiment (n = 32). A Hypothetical Learning Trajectory (HLT) was developed to structure learning activities around the critical analysis of authentic research excerpts. Analysis of students' written design proposals and clinical interviews revealed a conceptual progression from viewing interventions as true experiments to recognizing QEDs as contextually grounded compromises. Specifically, students demonstrated transformed reasoning by shifting from rigidly demanding random assignment to strategically utilizing intact groups to manage confounding variables. Retrospective analysis further highlighted the need for strengthened scaffolding regarding the notion of control. Consequently, the refined HLT yielded a proposed Local Instructional Theory (LIT) comprising four empirically grounded design principles: (1) contextualizing the need for comparison, (2) negotiating intact-group constraints, (3) evaluating threats to validity, and (4) reflecting on methodological compromises. In practice, this study provides mathematics teacher education programs with a structured framework to foster pre-service teachers' critical methodological reasoning, directly enhancing their capacity to conduct context-sensitive educational research.
##plugins.themes.bootstrap3.article.details##
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The author is responsible for acquiring the permission(s) to reproduce any copyrighted figures, tables, data, or text that are being used in the submitted paper. Authors should note that text quotations of more than 250 words from a published or copyrighted work will require grant of permission from the original publisher to reprint. The written permission letter(s) must be submitted together with the manuscript.References
Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013). Learning through case comparisons: A meta-analytic review. Educational Psychologist, 48(2), 87–113. https://doi.org/10.1080/00461520.2013.775712
Bakker, A. (2004). Design research in statistics education: On symbolizing and computer tools. Dissertation. Utrecht University. https://dspace.library.uu.nl/handle/1874/893
Bakker, A. (2018). Design research in education: A practical guide for early career researchers. Routledge. https://doi.org/10.4324/9780203701010
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1
Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3–15. https://doi.org/10.3102/0013189x034006003
Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical thinking and learning, 6(2), 81–89. https://doi.org/10.1207/s15327833mtl0602_1
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. https://doi.org/10.3102/0013189x032001009
Cole, R. (2024). Inter-rater reliability methods in qualitative case study research. Sociological Methods & Research, 53(4), 1944–1975. https://doi.org/10.1177/00491241231156971
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Houghton Mifflin Boston.
Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons. Journal of Policy Analysis and Management, 27(4), 724–750. https://doi.org/10.1002/pam.20375
Corbin, J., & Strauss, A. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory (4th ed.). Sage.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage publications.
Earley, M. A. (2014). A synthesis of the literature on research methods education. Teaching in Higher Education, 19(3), 242–253. https://doi.org/10.1080/13562517.2013.860105
Feldon, D. F. (2007). Cognitive load and classroom teaching: The double-edged sword of automaticity. Educational Psychologist, 42(3), 123–137. https://doi.org/10.1080/00461520701416173
Fisher, R. A. (1935). The design of experiments. Oliver & Boyd.
Freudenthal, H. (1991). Revisiting mathematics education: China lectures. Kluwer Academic Publishers.
Garfield, J. B., & Ben-Zvi, D. (2008). Developing students' statistical reasoning: Connecting research and teaching practice. Springer.
Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243. https://doi.org/10.3102/0091732x20903302
Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. Van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 29–63). Routledge. https://doi.org/10.4324/9780203088364-12
Gravemeijer, K. P. E. (1994). Developing realistic mathematics education. CDBeta Press.
Grosz, M. P. (2023). Should researchers make causal inferences and recommendations for practice on the basis of nonexperimental studies? Educational Psychology Review, 35(2), 57. https://doi.org/10.1007/s10648-023-09777-7
Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107. https://doi.org/10.1080/00461520701263368
Horton, N. J. (2023). Teaching causal inference: moving beyond ‘correlation does not imply causation’. Journal of Statistics and Data Science Education, 31(1), 1–3. https://doi.org/10.1080/26939169.2023.2178778
Johnson, R. B., & Christensen, L. (2020). Educational research: Quantitative, qualitative, and mixed approaches (7th ed.). SAGE Publications.
Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
Kuhn, D. (2010). Teaching and learning science as argument. Science Education, 94(5), 810–824. https://doi.org/10.1002/sce.20395
Martínez-Abad, F., & León, J. (2023). Inferencia causal en investigación educativa: Análisis de la causalidad en estudios observacionales de carácter transversal [Causal inference in educational research: Causal analysis in cross-sectional observational studies]. RELIEVE - Revista Electrónica de Investigación y Evaluación Educativa, 29(2), 3. https://doi.org/10.30827/relieve.v29i2.26843
Mumu, J., Prahmana, R. C. I., Sabariah, V., Tanujaya, B., Bawole, R., Warami, H., & Monim, H. O. L. (2021). Students' ability to solve mathematical problems in the context of environmental issues. Mathematics Teaching Research Journal, 13(4), 99–111.
Mumu, J., & Tanujaya, B. (2019). Analysis of mathematical connection in abstract algebra. Journal of Physics: Conference Series, 1321(2), 022105. https://doi.org/10.1088/1742-6596/1321/2/022105
Osborne, J. (2014). Scientific practices and inquiry in the science classroom. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (Vol. 2, pp. 579–599). Routledge.
Plomp, T. (2013). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), Educational design research (pp. 10–51). Netherlands Institute for Curriculum Development (SLO).
Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211–227. https://doi.org/10.1002/sce.3730660207
Prahmana, R. C. I. (2017). Designing mathematics learning trajectory: An introduction. Lambert Academic Publishing.
Prediger, S., Gravemeijer, K., & Confrey, J. (2015). Design research with a focus on learning processes: an overview on achievements and challenges. Zdm, 47(6), 877–891. https://doi.org/10.1007/s11858-015-0722-3
Pyott, L. (2021). Tennis anyone? Teaching experimental design by designing and executing a tennis ball experiment. Journal of Statistics and Data Science Education, 29(1), 22–26. https://doi.org/10.1080/10691898.2020.1854638
Rahayu, W., Tanujaya, B., & Iriyadi, D. (2022). Desain eksperimen untuk bidang pendidikan [Experimental design for education]. Jakad Media Publishing.
Reichardt, C. S. (2019). Quasi-experimentation: A guide to design and analysis. Guilford Publications.
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522. https://doi.org/10.1207/s1532690xci1604_4
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Siemon, D. (2021). Learning progressions/trajectories in mathematics: Supporting reform at scale. Australian Journal of Education, 65(3), 227–247. https://doi.org/10.1177/00049441211045745
Simon, M. A. (1995). Reconstructing mathematics pedagogy from a constructivist perspective. Journal for Research in Mathematics Education, 26(2), 114–145. https://doi.org/10.5951/jresematheduc.26.2.0114
Smucker, B. J., Stevens, N. T., Asscher, J., & Goos, P. (2023). Profiles in the teaching of experimental design and analysis. Journal of Statistics and Data Science Education, 31(3), 211–224. https://doi.org/10.1080/26939169.2023.2205907
Tanujaya, B. (2013). Penelitian percobaan [Experimental research]. Remaja Rosdakarya.
Tanujaya, B., & Mumu, J. (2020). Students’ misconception of HOTS problems in teaching and learning of mathematics. Journal of Physics: Conference Series, 1657(1), 012081. https://doi.org/10.1088/1742-6596/1657/1/012081
Tanujaya, B., Prahmana, R. C. I., & Mumu, J. (2018). Designing learning activities on conditional probability. Journal of Physics: Conference Series, 1088(1), 012087. https://doi.org/10.1088/1742-6596/1088/1/012087
ten Hove, D., Jorgensen, T. D., & van der Ark, L. A. (2024). Updated guidelines on selecting an intraclass correlation coefficient for interrater reliability, with applications to incomplete observational designs. Psychological Methods, 29(5), 967–979. https://doi.org/10.1037/met0000516
Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Educational design research. Routledge. https://doi.org/10.4324/9780203088364
Vosniadou, S. (2013). International handbook of research on conceptual change (2nd ed.). Routledge.